Improving Digital Library Experiences and Support with Online Research Guides 


135 


Improving Digital Library Experiences and 
Support with Online Research Guides 

Laura Brewer, Holly Rick and Karen A. Grondin 
University of Phoenix 

Abstract 

With a goal of improving the development and delivery of effective online information literacy 
resources, the purpose of this study was to look at how program level and the timing of the 
introduction of a Literature Review library guide within the program influenced online business 
student perceived value of the resource. A population of undergraduate business students (N=355) 
and online MBA students (N=319) were introduced to a Literature Review library guide during 
specific points in their programs. Students were asked to complete an online survey that included 
17 closed-ended items designed to measure perceived usefulness, satisfaction and likeliness to use 
the guide again. The survey also included two open-ended questions asking students to discuss 
those elements of the guide they found most valuable and whether they wanted any additional 
features included in the guide. The data collection strategy required faculty post information about 
the Literature Review library guide and the survey in their courses at two specified times in the 
course. A low response rate (3.5%) may have resulted from inconsistencies in how faculty shared 
infonnation about the guide and the study in their courses. Although the small sample size (n=24) 
limited the planned analysis, and results indicated no statistical significance between groups, 
descriptive findings were reported, and trends were used to revise the resource and inform future 
development of library research guides. Overall, students reported being satisfied with the 
resource and found it usable. Graduate students were more likely to report elements of the guide 
that supported effective search and evaluation strategies were valuable; whereas, undergraduate 
students tended to value the links to writing resources. Student feedback also suggested that the 
earlier the guide was introduced in the program, the more likely students would use the resource. 
Adopting a model that embeds resource guides early in academic programs and aligns guide 
content with the curriculum should lead to increased use of the resource. 
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Introduction 

The traditional role of a library has been to provide crucial resources and services for 
student needs beyond providing books and serials. While web-based content and electronic media 
are available at students’ fingertips, the web-based content may not provide the support that 
students need to complete their programs. Students struggle to find scholarly resources to support 
their academic work. For online students, these struggles can become overwhelming. Many 
students fall into search habits that may assist them in completing an assignment, but will not build 
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lifelong learning skills or infonnation literacy skills needed for future employment. Digital 
libraries play an important role for students; librarians are educators that procure content and 
provide instruction on how to use and evaluate resources. 

University libraries have been slowly building robust digital collections to support their 
students. Digital library collections can be difficult to navigate, and many are developed on an 
infrastructure that is not intuitive to those conducting research. Cothran (2011) strongly argues 
for “librarians to focus on improving the usability and accessibility of library resources by linking 
library databases and Google Scholar, instead of simply discouraging students’ Google Scholar 
usage” (p. 298). Without some intervention with a library-trained professional, students may find 
obstacles accessing library content. 

Not all libraries offer for credit courses or provide embedded librarians to support the 
curriculum. Academic libraries have been exploring new ways to include library instruction that 
engages students during their times of need. By building course materials that support the learning 
process and providing infonnation literacy instruction in the learning management system (LMS), 
students should have a better experience using the library resources, be more likely to find the 
appropriate library content, and build information literacy skills necessary for lifelong learning. 
Librarians will be better able to strategically build content that supports the online learner if they 
understand students’ perceptions of the resources. 

Buehler (2004), Shank and Dewald (2003), and Bowen (2012) have identified value in 
embedding library content in the LMS system, which brings the library to the students and faculty 
where they “work.” Improving the library’s connection with other university systems, like the 
LMS, can result in an increased return on investment in the library collection. The work completed 
by Buehler (2004), Shank and Dewald (2003), and Bowen (2012) did not did not examine students’ 
perceptions of the library content or the role of timing when students will get access to library 
instruction and content. 

If librarians are not able to be in the classroom to teach information literacy skills, students 
will likely resort to their previous information seeking skills, which may not lead to locating the 
most appropriate resources to support their work. Infonnation literacy support and materials must 
be embedded within the curriculum to support student learning and to be recognized and prioritized 
in academic plans. As noted by Khan and Qutab (2016) “The librarians must understand that users 
always know their information needs, but they do not know where to find these information” (p. 
312). 

Librarians are required to create materials that students find supportive, useful, and that 
students will value for their research tasks. When students cannot locate or access the material 
they need, they can become disenfranchised. Factors that impact how students perceive online 
materials include slow downloads, difficulty reading online due to other technology options such 
as games, a distraction from social media, and emails. Students tend to use specific journals 
because of familiarity. By understanding how online students make meaning about their 
experiences with the library and the library’s resources, librarians and academic affairs personnel 
can better promote online student learning. 

If students do not believe that a resource is valuable, they will not use that resource. Wu 
and Chen (2012) found graduate students often use library resources and that graduate students 
recognize library resources are important for their classwork. Students can be confused about how 
to find information and what resources may be needed to complete their work; the library provides 
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students with more than subscription databases, books, and serials. Resources that explore topics, 
present information literacy in a new way, or support an assignment can be perceived as valuable 
to students in their research process. 

Early work by Beagle (2000) identified that librarians need to take an active role in the 
instructional design of infonnation literacy teaching and resource creation. Additional works by 
Gilbert, Knutson, and Gilbert (2012), Ouellette (2011), and Liu and Luo (2011) indicated student 
perceptions of the library resources impacts student learning, collection development, and 
suggested patterns related to how students engage and perceive resources. One of the ways the 
library takes an active role is to assess the unique needs of their student body. 

This study was conducted at a large institution with a diverse, primarily online population. 
According to the National Center for Education Statistics (2015), as of Fall 2015, 165,743 students 
were enrolled; 133,211 were undergraduate students and 32,532 were graduate students. Ninety- 
five percent of graduate and ninety-eight percent of undergraduate students were enrolled solely 
in distance education. The University has eleven colleges and schools. The Business programs 
graduate more students than any other University program with 4,393 graduating with a Bachelor’s 
in Business Administration and Management and 2,772 graduating with an MBA in 2015. The 
Business programs were selected for this study because they represent the largest group of students 
and thus the resources had the potential to have to benefit the greatest number of students while 
under study. Each ground campus at the University has a Student Resource Center that provides 
assistance with writing and mathematics, while University Library does not have a ground 
presence and is entirely virtual. The University resources used in this study, including the writing 
resources, were all completely online resources. 

This study was designed to obtain a descriptive evaluation of online students’ experiences 
with a University library resource designed to support their research activities. Researchers 
created a survey to capture students’ perceived usefulness, satisfaction and likeliness to use the 
library resource provided in their course. Data was collected without student identifiers to promote 
honest feedback. The remainder of this paper will include a brief literature review, the research 
questions guiding the study, as well as the method of data collection and results. Implications for 
further research will be discussed. 


Literature Review 

Murray (2015), Catalano (2016), and Haddow (2012) have shown that having a librarian 
work with students increases retention and increases the information literacy benchmark for the 
University. When moving curriculum to an online platform regardless of the learning management 
system (LMS), the ability to integrate a librarian in the classroom experience can become almost 
impossible. “Research in virtual reference environments reveals that leading factors for non-use 
include unawareness of the service, satisfaction with other information sources, and lack of 
confidence in the chat librarian’s ability” (Liu & Luo, 2011, p. 231). While librarians and faculty 
build many different types of instructional materials and resources to support their students, they 
may not look at how the resource is being used, where the resource is placed within the course or 
program, or care about how often students use the resource. 

Historically, librarians have developed and made available library guides and pathfinders 
as a service to library patrons and to support information literacy instruction. While library guides 
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can take the form of paper handouts or electronic tools, it is important to understand what library 
guide characteristics students perceive as valuable to their academic experiences and success. 
Students from a variety of backgrounds come to the University with diverse experiences that may 
influence their library research skills. Librarians build guides without having information about 
students’ previous experience in using the library or baseline of their information literacy skills. 
Librarians also aim to design the guides to support multiple learning styles. Bellard (2007) 
identified that while the majority of graduate students self-reported their library skills as fair or 
better, many could not tell the differences between a library catalog and a database and were also 
unable to identify subject specific databases. Bellard’s study offered the participants an optional 
library workshop, and the students who attended the workshop reported a higher comfort level 
with library resources and a majority of the student participants felt that library instruction should 
be part of the curriculum. 

Green and Browser (2002) found that a collaborative teaching environment including a 
faculty librarian helped reduce graduate student anxiety. Students felt that the faculty and the 
librarian each had unique areas of expertise that were helpful as they went through the dissertation 
process. Both Rempel (2010) and Green and Browser (2002) found that students benefit from 
librarian involvement, especially when working on the literature review. 

Using Guides 

Research around the utilization of library guides has identified that when library guides are 
used, these guides do improve research skills, GPA and retention (May & Leighton, 2013; 
Wakeham, Roberts, Shelley, & Wells, 2012). However, most utilization studies have focused on 
how the library guides are built and embedded (May & Leighton, 2013; Wakeham, Roberts, 
Shelley, & Wells, 2012), and not on how student perception of the guide may relate to student 
decisions to use the resources. Previous research also has not examined how student perceptions 
might be influenced by when an online library resource is introduced within a course or program. 

Timing 

An important piece of planning for library research skills instruction is the timing of the 
instruction. Many studies have looked at the benefits of offering instruction at the student's’ point 
of need. Rempel (2010) conducted a longitudinal study of graduate students who attended a 
literature review workshop presented by the library. The workshop was offered to students when 
they were new to their program, and the students found the workshop was an effective use of time. 
It is also important to consider programmatic requirements when determining the point of need. 
For example, Rempel (2010) and Neves and Dooley (2011) point out those students who are 
required to submit a project proposal benefit from library instruction at an earlier time than those 
who do not have this requirement. Additionally, students who had to come up with a topic tended 
to start their literature review almost a full year after starting their graduate research. Those 
students would probably benefit from library instruction at a later time. Mahaffey (2012) 
concluded that students valued having a research guide when they needed the resource. 

Liu and Luo (2011) focused on how often graduate and undergraduate students used their 
digital library. Their research showed that graduate students requested earlier access to the online 
library and content to ensure that they were current in the field. The undergraduate students in this 
study noted that the library was difficult to use and the material was older. This discrepancy in 
experiences based on student level may be explained by under-developed research skills of 
undergraduate students or a lack of complexity in their research needs. 
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Perception 

Most of the perception research on online library guides revolves around the perceptions 
between print versus electronic resources (Liu, 2006, Lombardo & Miree 2003). Other perception 
research has been completed on the adequacy of the library resources for both ground and online 
(Spahr, 2015). Student and faculty perception data can be helpful in assessing library instruction, 
materials or to re-evaluate services. However, additional information about student satisfaction 
with the timing of the introduction of the library resources may also support informed decisions 
about information literacy instruction in the online environment. 

Theory 

A social constructivist framework suggests learning is constructed through social 
interactions (Vygotsky, 1978). For this study, online students interact with the information literacy 
resource, a library guide about literature reviews, and students make meaning of these interactions. 
The meanings that online students attribute to their experiences interacting with the literature 
review library guide will likely influence their future interactions with the University Library. If 
we better understand online student experiences with the University Library, and with this 
particular online library resource, we may be better able to support the development of effective 
online information literacy resources. 

Research Questions and Hypotheses 

This study was directed by two research questions, and each question was associated with 
a hypothesis. 

• RQ1: How does program level influence online students’ perceived value of a “Literature 
Review” online library guide? 

• HI: There will be no significant difference between undergraduate business and MBA 
students’ perceptions of the value the “Literature Review” online library guide. 

• RQ2: How does the timing in which the “Literature Review” online library guide is 
introduced to students influence their perceived value of the resource? 

• H2: The earlier the “Literature Review” online library guide is introduced to students, the 
greater the students will value the resource. 


Method 

Population and Sample 

The population for this study is online undergraduate business students and online MBA 
students. Online undergraduate students were solicited from one research course introduced early 
in the program (N=83) and from the program’s capstone course (N=561). The MBA sample was 
recruited from a research course introduced early in the program (N=232) and from the program’s 
capstone course (N=314). The purposive sample consisted of all students enrolled in all sections 
of these four courses (N=l 190) with start dates during September and October 2016. Enrollment 
numbers were calculated by those students who were included on the final class rosters after the 
School’s drop/add dates for these courses. 

Method of Data Collection 

Data was collected using online surveys made available to students with the Literature 
Review library guide within the four business courses. The online survey included 17 closed- 
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ended items that had been used by the University Library to assess usability and design for other 
projects. These items were revised slightly to measure student experiences with the Literature 
Review library guide. The 17 items were designed to measure perceived usefulness, satisfaction 
and likeliness to use again using 5-point Likert scales where the higher the value indicated the 
greater the usefulness, satisfaction, and likeliness to use again. Also, two open-ended questions 
were asked of all students: 

• What element of the Literature Review library guide do you think was most valuable and 
why? 

• Is there anything you’d like to see included in Literature Review library guide that was not 
currently part of the resource? 

All faculty teaching these four courses with start dates between September 13 and October 
18, 2016, were sent an email before their section start date informing them about the Literature 
Review library guide and study. The faculty had not seen the resource before this initial email. 
The email was sent from the School of Business leadership and asked the teaching faculty to create 
two announcements in their online course sites. A template for both announcements was provided. 
The announcement explained that the University Library was piloting the new online library 
resource in this particular course and the resources were designed by the University Library to 
support students with the development of a literature review. The announcement went on to 
explain that the resource was an optional supplement to the course and that interested students who 
chose to use the resource would also be asked to voluntarily complete a short survey to provide 
feedback about their experience using the resource. The announcement also included a link to the 
short 5 to 10-minute online survey. The online business courses are five weeks in length and 
faculty were asked to post two announcements about the guide and the survey in their classes 
during week 2 and week 5. 

The study was designed to manage concerns about human research subjects. Participation 
was voluntary and a standard infonned consent document was included before students gaining 
access to the online survey. In addition to using anonymous survey URLs, researchers asked to 
waive documentation of infonned consent as a mechanism to maintain the anonymity of all student 
participants. 

Limitations with Response 

A few issues impacted student response patterns. The researchers developed the text for 
the faculty course announcements about the Literature Review library guide and the associated 
student survey, as well as instructions for faculty with regards to how and when to share the 
infonnation with their classes. However, the researchers had to rely on department administrators 
to share this information with faculty and for faculty to post the announcements in their classes on 
schedule. A review of class websites indicated many of the faculty did not post one or both of the 
announcements (Table 1). 

While there were 1190 students enrolled in these courses, only 684 received at least one 
notice about the Literature Review library guide, and only 226 students received both scheduled 
notices. The overall response across courses and including students enrolled in courses where at 
least one notice was posted was only was 3.5% (n=24). Response varied by course, with the 
highest response from students enrolled in the early undergraduate course (14%) and the lowest 
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rate of response coming from students enrolled in the undergraduate later course (n=l). A more 
thorough examination of response patterns did show that in courses where the faculty member did 
post two announcements, there was a higher response rate (Table 1). 


Cour 

ses 

Final 

Roster 

# 

Students 

Either 

Notice 

# 

Students 

Both 

Notices 

% Students 
Receiving 
Both 
Notices 

Total # 
Response 

Total % 
Response 

Total % 
Response 
Students 
Receiving 
Both 

U-E 

83 

70 

39 

47% 

12 

14% 

31% 

G-E 

232 

133 

47 

20% 

6 

3% 

13% 

U-L 

561 

295 

89 

16% 

1 

0% 

1% 

G-L 

314 

186 

51 

16% 

5 

2% 

10% 


Table 1. Recruitment and Response Patterns. “U” is the “undergraduate” courses and “G” is the 
“graduate” courses. “E” is the “early” courses and “L” is the “late” courses. 


Since access to the Literature Review library guide resource was only made available 
through the faculty announcements, it was not surprising to see that data from the library guide’s 
analytics also showed somewhat higher resource views when the faculty shared the infonnation 
with their students (see Figure 1). It also appeared that page hits were higher after the first notice 
than after the second notice. The early increased number of resource page hits might have also 
been related to faculty accessing the resource themselves before posting the information in their 
classes. It was impossible to connect page hits to individual students, faculty or identified courses 
in this study. 

Method of Data Analysis 

Initially, descriptive statistics were run on all items. Limited response and small group 
sample sizes (see Table 1) precluded hypothesis testing and as a result, the findings presented 
focuses on response frequencies when examining group differences. 
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Figure 1. Resource Views and Survey Response 


Results 

Although the study was designed to examine mean differences between groups 
(undergraduate students versus graduate students and students enrolled in courses early in the 
program versus student enrolled in later courses), the low response rate resulted in no statistical 
significance between groups. Findings were examined for descriptive purposes and the trends, 
though not statistically significant, were used to revise the resource and to inform future 
development of University Library guides. 

Satisfaction with Resource 

Across undergraduate and graduate business research courses, and across courses that were 
taken early in the programs and courses that were taken later in the programs, students indicated 
they were very satisfied with the Literature Review library guide (A=4.08) and that they would 
very likely use the Literature Review library guide again (A=4.04). 

There were almost no differences in how undergraduate and graduate students rated how 
likely they were to use the Literature Review library guide again (undergraduate A=4.08, graduate 
A=4.00) or how satisfied they were with their experience with the Literature Review library guide 
(undergraduate X=4A5, graduate X=4.00). 

Perception of Usability 

Similarly, students across levels and courses indicated they were satisfied with the way the 
Literature Review library guide perfonned (X=4.22), the look and feel of the guide (A=4.09), 
navigation (A=4.00) and the design intuitiveness (A=3.96). Students across levels and courses 
also tended to agree that the Literature Review library guide was easy to use (Z=3.78), that they 












Improving Digital Library Experiences and Support with Online Research Guides 


143 


imagined most people would learn to use the guide very quickly (A=3.65), and that they found the 
various elements of the guide well integrated (A=3.58). 


I would imagine that most people would learn to use 
Literature Review Library Guide very quickly. 

I found the various elements of the Literature Review 
Library Guide well integrated. 



3.S4 


I thought the Literature Review Library Guide design 
was easy to use. 



4.18 


System Performance . ■; 


2S 


Look and Feel 


4.09 

4.08 


Intuitiveness 


3.82 


4.08 


Navigation 


3.82 


4.1? 


**Grad 
r Undergrad 


Figure 2. Usability Items - Undergraduate vs. Graduate Students (means reported) 


Examining patterns in perceived usability across the undergraduate and graduate business 
students suggest minimal differences except with regards to the question about how easy the 
Literature Review library guide was to use, where the undergraduates were more likely to agree 
(A=4.00) as compared to the graduate students (A=3.55). Similar patterns were found around the 
questions about perceived intuitiveness (undergraduate A=4.08, graduate A=3.82) and about the 
design and navigation (undergraduate X=4. 17, graduate =3.82) of the resource. 

When asked about what students found most valuable about the Literature Review library 
guide, undergraduate students reported, “The layout of the site was very easy to read and navigate” 
and “Everything is pretty easy. No complaints so far.” Some graduate students reported similar 
comments, such as explaining what they felt was most valuable was “The ease of information” 
and “The ease of use...because it means it will be user-friendly and more people will be able to use 
it as well.” However, one graduate student explained when asked what could be improved about 
the site, “A direct link would be most efficient. The way I was taught to access the resource was 
like a treasure map.” And another graduate student explained, “I would like more infonnation 
explaining what each of the databases is and what it is best used for.” 

Value of Content 

Students across levels and courses also reported they would likely use the Literature 
Review library guide often (A=3.83). When asked whether they would use the various resources 
within the Literature Review library guide, students tended to agree that they would likely often 
use the Managing References (A=4.24), the Writing Resources (X=4. 19), the Evaluate Sources 
(A=4.00), the Develop a Search Strategy (A=4.00) and the Start Writing (X=3.95) resources. 
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Although all students tended to indicate they found the resources in the Literature Review 
library guide important by explaining that they would likely use the features often, when 
examining differences between undergraduate and graduate students on their perceptions of the 
specific resources included in the Literature Review library guide, graduate students appeared to 
value a few of the writing and reference resources more highly than their undergraduate peers. For 
example, graduate students were more likely to report that they would often use the Writing 
Resources (graduate A=4.56, undergraduate X=3.92) and the Managing References (graduate 
A=4.50, undergraduate X=4.00) (Figure 3). 


I would use trie Literature Review Library Guide often. 


Writing Resources 


Start Writing 


Manage References 


Evaluate Sources 


Develop a Search Strategy 



4.10 


4.10 


4.56 


4.50 


*Grad 
; Undergrad 


Figure 3. Content Items - Undergraduate vs. Graduate Students (means reported) 


Undergraduate students indicated that specific features from the University Writing Center, 
which was available from the Literature Review library guide, were helpful. For example, one 
student said, “...I think the center of writing excellence has helped me out the most”. Another 
undergraduate said, “I like to have access to the APA template,” and another suggested, “The 
resource management is the most valuable to me it helps you cite resources... Awesome!”. In 
contrast, the graduate students focused on elements related to searching the literature. For 
example, one graduate student reported that they found the most valuable feature to be to “Evaluate 
sources, it allowed me to find credible sources.” Another graduate student explained: “The most 
valuable element was Develop a Search Strategy. Assignments can have thousands of articles, 
books, etc. to reference. If you educated on how to narrow and redefine your search, you increase 
your chances of locating relevant data.” 

Timing and Literature Review Library Guide 

This study was designed to examine whether the timing of introduction of the Literature 
Review library guide into the academic programs was related to students’ reported use of the guide 
and their satisfaction with the guide. Timing was considered in relation to courses (early in the 
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program and later in the program) as well as to programs (undergraduate business and graduate 
business). 

When looking at response patterns related to the timing of the introduction of the Literature 
Review library guide, those students enrolled in courses earlier in their program were much more 
likely to indicate they would use the resource again (early A=4.17, late X=3.67) and that they were 
satisfied with their experience (early X=4. 17, late W=3.83). Because the response set was so small 
and because there was only one undergraduate response from the later course, it should be noted 
that the “late” respondents were nearly all graduate students. The conflation of graduate student 
level with the “later” course response should be considered when looking at the comparison of 
early students and late students on the specific usability and content items in this study. 

All students were asked how likely it was that they would use the Literature Review library 
guide had it been introduced at a different point in their degree program. Students enrolled in the 
earlier courses indicated that they were less likely (A=3.61) to use the resource more had it been 
introduced later in the course. In contrast, the students enrolled in the later courses indicated they 
were highly likely (A=4.50) to use the resource more had it been introduced earlier in their 
program. The timing of the introduction of the Literature Review library guide within academic 
programs may be related to whether or not students use the resource. In particular, the response 
patterns suggest that students would like to be introduced to this resource earlier in their academic 
programs. One graduate student enrolled in the later research course explained: 

This does not answer your question, but I wanted to provide additional feedback. I 
would suggest this guide is required resource students should review. I am 
currently in my last course and have sent questions to the librarians and found their 
responses very helpful. If I were aware of this resource, I would have taken time 
to utilize it and apply suggestions it provides. 


Discussion 

Guide Design 

While both undergraduate and graduate students expressed satisfaction with the guide and 
indicated that they would use it again, the study has identified the need to revise the guide based 
on students’ perceptions of usability. The fact that ease of use, ability to quickly learn how to use 
the guide and integration of elements were all rated slightly lower than other variables of the guide 
point to areas for improvement. Further development of the Literature Review library guide will 
be completed with additional usability testing to ensure that the guide meets students’ preferences 
for information literacy framework with a clean design, consistent and meaningful language, and 
a limited number of links and pages on those guides (Ouellette, 2001). While keeping these 
preferences in mind, ensuring that the guides have the appropriate content that meets the 
curriculum outcomes for infonnation literacy instruction will need to be assessed. Finding the 
balance between providing enough information and avoiding the tendency to overwhelm students 
with too much information will require additional student feedback, curriculum design, and 
mapping of curriculum outcomes. A good method to support this might be to follow Ouellette’s 
(2001) recommendation to break guides down into sub-disciplines. While the Literature Review 
library guide is not associated with a particular discipline, and it is focused specifically on 
resources for completing a literature review, the guide could be further streamlined and broken 
down into “bite-sized” guides and embedded at students’ point of need. 
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Only minimal differences between undergraduate and graduate business students’ 
perceptions of usability of the guide were found. However, graduate students reported ease of use 
and intuitiveness and navigation slightly lower than undergraduates, and this pattern may be due 
to different expectations on the part of the two student groups as well as different academic needs. 
Future research could explore this area further. Also, a future approach to addressing unique 
undergraduate and graduate student needs may involve building two separate Literature Review 
library guides, one for an undergraduate audience and the other for a graduate audience. Using a 
service design approach to guide development will help identify if separate guides are a better 
option, help create guides that are more supportive of differing student needs, and better contribute 
to the success of these two groups of students (German, 2017). 

Developing a better understanding of the base knowledge, needs, and expectations of 
different groups of students will help with future guide design. Students place a high value on 
electronic resources and are often confused by the language that librarians use (Ouellette 2001, 
Wu & Chen, 2012). Given that students may not understand what resources are available in their 
library, or how to navigate to those resources, guides should be designed with the goal of 
supporting student needs. For example, libraries should aim to make electronic resources easily 
noticeable, make them easy to find, and clearly describe resource content. Knowing that students 
are more likely to use a resource that is recommended by faculty, and that they are more likely to 
report greater success and satisfaction when faculty or a librarian recommended that resource 
specifically for an assignment, will inform future guide development as well as placement of the 
guide (Ouellette, 2001 & Spahr, 2015). 

Content Relevance 

Comparing undergraduate and graduate business students’ responses around the value of 
content also suggests a potential need to model this guide differently for undergraduate and 
graduate students. Graduate students were more likely to report that they would use the Writing 
Resources and Managing References pages often than undergraduates. Undergraduate students 
indicated in the open-ended question that the content of the University Writing Center page was 
helpful. A possible reason for this is the undergraduate students are still learning the process of 
writing, while graduate students conceivably need less help in this area. In contrast, the graduate 
students focus on the elements related to searching the literature such as content on how to evaluate 
sources and how to develop a search strategy. 

Timing of Guide Introduction 

The fact that students who were introduced to the guide in courses later in their program 
reported that they were highly likely to have used the guide had it been introduced earlier in their 
program was unsurprising and supports the results indicating the students found the guide valuable. 
It is also not surprising that students who were introduced to the guide early in their program were 
less likely to report that they would use the guide more had it been introduced at a later time. The 
students recognized that this guide provided more value at certain times in their program and that 
they would have had a greater need for the guide earlier in their program. Rempel (2010), Neves 
and Dooley (2011), Mahaffey (2012), and Shank and Dewald (2010) identified that the earlier 
infonnation literacy instruction is introduced and more often the students are exposed to 
infonnation literacy concepts, the higher the probability the students will apply the skills in their 
coursework. The library guides need to be embedded at the appropriate time. “The closer the link 
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between course assignments and library resources to help with those assignments, the greater 
likelihood that students will access library information” (Shank & Dewald, 2003, p. 41). 

One way to help determine if a guide is embedded at the correct point when students need 
the information is to look at the use analytics. The LibGuides platform provides data on the 
number of views at the guide, page, and link or asset level. It would be valuable to know the 
number of unique user views, repeat visitors, and session length to gain a more comprehensive 
understanding of use. Google Analytics is one tool that can provide this data and this tool will be 
integrated for future evaluation of guide use. If the number of unique visitors increases shortly 
after a guide is introduced in the curriculum, this can indicate that the guide is findable. Looking 
at data for repeat visitors can help determine if the content of the guide is valuable and meeting 
student learning outcomes. 

Future Research 

Future research on course specific guides will be conducted with continued evaluation of 
student perceptions of usability, the value of the content, and timing of guide introduction while 
additionally evaluating usage. These guides will be built for doctoral studies supporting research 
methods, supporting doctoral writing, and publication processes for their doctoral work. This 
study will include additional data collection points such as student end of course surveys, faculty 
end of course surveys, and student data (grades, continued course enrollment, and GPA). By 
engaging the faculty in the process, we are looking at having more student use of the project. 

Students noted that they would have used the guides more if they were introduced earlier 
within their program. Future work needs to include identification of library resource needs, 
identification of when in a program the resource is introduced, and where the resource is embedded 
within the structure of the course. 

After the completion of this initial study, the University Library has begun building library 
guides that are embedded in the first six courses that are taken by all undergraduate students. The 
guides are being built to support signature assessment assignments in a series of courses developed 
to promote student success. These guides fit the definition of Micro-Level Library Courseware 
Involvement (MiLLCI) as described by Shenk and Dewald (2003). MiLLCI can be advantageous 
by allowing student access to focused library content at the point of need within the LMS. 
Embedding the guides directly into the curriculum should increase use of the guides and the 
associated library resources. 

Library resource design can benefit from ongoing, gradual user experience testing 
(Pennington, 2015; Sonsteby & DeJonghe, 2013; Tidal, 2012). Usability design can help librarians 
develop awareness of the problems that their students encounter and help them build library 
resources more focused on their students' needs (Pennington, 2015; Sonsteby & DeJonghe, 2013; 
Tidal, 2012). These findings suggest that additional user testing of the library guides should be 
completed. As changes are made to the design of the library pages, it is advantageous to make 
sure that the students are still able to follow steps to find resources on the library guide page with 
minimal disruption. Future research will involve completing usability tests with small groups of 
students who will be impacted by the design changes and then ask these students to show their 
steps, and the researchers will observe what they do. By conducting this research with students 
with different demographics, additional customization of the library guide to meet individual 
student needs. 
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